1. Field of the Invention
Embodiments of the present invention generally relate to flexible region based sample adaptive offset (SAO) and adaptive loop filter (ALF) in video coding.
2. Description of the Related Art
The Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T WP3/16 and ISO/IEC JTC 1/SC 29/WG 11 is currently developing the next-generation video coding standard referred to as High Efficiency Video Coding (HEVC). Similar to previous video coding standards such as H.264/AVC, HEVC is based on a hybrid coding scheme using block-based prediction and transform coding. First, the input signal is split into rectangular blocks that are predicted from the previously decoded data by either motion compensated (inter) prediction or intra prediction. The resulting prediction error is coded by applying block transforms based on an integer approximation of the discrete cosine transform, which is followed by quantization and coding of the transform coefficients. While H.264/AVC divides a picture into fixed size macroblocks of 16×16 samples, HEVC divides a picture into largest coding units (LCUs), of 16×16, 32×32 or 64×64 samples. The LCUs may be further divided into smaller blocks, i.e., coding units (CU), using a quad-tree structure. A CU may be split further into prediction units (PUs) and transform units (TUs). The size of the transforms used in prediction error coding can vary from 4×4 to 32×32 samples, thus allowing larger transforms than in H.264/AVC, which uses 4×4 and 8×8 transforms. As the optimal size of the above mentioned blocks typically depends on the picture content, the reconstructed picture is composed of blocks of various sizes, each block being coded using an individual prediction mode and the prediction error transform.
In a coding scheme that uses block-based prediction, transform coding, and quantization, some characteristics of the compressed video data may differ from the original video data. For example, discontinuities referred to as blocking artifacts can occur in the reconstructed signal at block boundaries. Further, the intensity of the compressed video data may be shifted. Such intensity shift may also cause visual impairments or artifacts. To help reduce such artifacts in decompressed video, the emerging HEVC standard defines three in-loop filters: a deblocking filter to reduce blocking artifacts, a sample adaptive offset filter (SAO) to reduce distortion caused by intensity shift, and an adaptive loop filter (ALF) to minimize the mean squared error (MSE) between reconstructed video and original video. These filters may be applied sequentially, and, depending on the configuration, the SAO and ALF loop filters may be applied to the output of the deblocking filter.